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Virtual  Representation  of  IID  Observations  in 
Bayesian  Belief  Networks 


Abstract 

Local  computation  for  updating  Bayesian  belief  networks  proceeds  in  the 
context  of  a  “join  tree,”  consisting  of  subsets  of  interrelated  variables 
(cliques)  joined  by  their  intersection  sets  in  a  singly-connected  graphical 
structure.  When  multiple  independent  and  identically-distributed  (IID) 
observations  of  a  variable  can  be  made,  identically  structured  cliques 
corresponding  to  each  potential  observation  appear  as  terminal  nodes  in  the 
join  tree.  This  note  shows  how  it  is  possible  to  absorb  information  from  an 
indefinite  number  of  observations  of  this  type  without  preconstructing  and 
manipulating  cliques  for  all  potential  observations.  An  “update  &  replace” 
strategy  carries  the  necessary  information  with  only  two  nodes  for  a  family 
of  IID  observations  of  a  variable  at  any  point  in  time. 

Key  words:  Bayesian  inference  networks,  causal  probability  networks, 
expen  systems,  influence  diagrams,  intelligent  tutoring 
systems,  local  computation. 


Introduction 


Bayesian  inference  networks  (also  referred  to  as  causal  probability  networks  and 
influence  diagrams)  are  systems  for  representing,  and  carrying  out  probability-based 
inference  in,  assemblages  of  interrelated  variables  (Andreassen,  Jensen,  &  Olesen,  1990; 
Lauritzen  &  Spiegelhalter,  1988;  Pearl,  1988;  Shafer  &  Shenoy,  1988).  Starting  from  a 
directed  acyclic  graphical  (DAG)  representation  of  the  conditional  independence 
relationships  among  the  variables,  a  representation  of  the  joint  probability  distribution  is 
constructed  in  terms  of  products  of  the  distributions  of  subsets  of  interrelated  variables 
(cliques),  divided  by  distributions  of  intersecting  subsets  (clique  intersections).  A 
corresponding  “join  tree”  representation  of  cliques  with  the  property  of  single-connectness 
enables  coherent  propagation  of  the  consequences  of  new  information  throughout  die 
network,  employing  only  calculations  local  to  cliques  and  their  immediate  neighbors. 
Commerically  available  computer  programs  for  structuring  and  using  Bayesian  inference 
include  ERGO  (Noetic  Systems,  Inc.,  1991)  and  HUGIN  (Andersen,  Jensen,  Olesen,  & 
Jensen,  1989). 

This  note  presents  a  technique  for  reducing  the  effective  computing  size  of  Bayesian 
inference  networks  with  nodes  that  represent  repeated  independent  and  identically- 
distributed  (HD)  observations.  It  is  assumed  that  all  such  variables  in  the  class  have  exactly 
the  same  parents  and  the  same  conditional  probabilities,  given  values  of  their  parents. 
Examples  are  as  follows: 

•  In  target  shooting,  repeated  shots  at  the  same  target  are  made  under  the  same 
conditions.  The  IID  observed  variables  are  individual  shot  scores,  and  the 
shooter’s  level  of  skill  is  the  parent.  Successive  observed  shot  scores  are  of  interest 
only  insofar  as  they  cumulatively  provide  increasing  precision  about  the  shooter’s 
level  of  skill. 

•  In  medical  diagnosis,  unreliable  tests  may  be  repeated  to  increase  accuracy. 
Assuming  the  “true  value”  remains  constant,  the  repeated  test  results  are  HD,  with 
conditional  probability  distributions  determined  by  the  true  physiological  stare  and 
the  common  distribution  of  the  measurement  errors. 

•  In  educational  testing,  item  responses  may  be  sequentially  obtained  to  randomly- 
sampled  items  from  a  domain.  The  parent,  and  the  variable  of  ultimate  interest,  is  a 
student’s  probability  of  correctly  answering  a  randomly-selected  item;  that  is,  the 
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student’s  “true  score.’’  Item  responses  are  modeled  as  conditionally  independent 
Bernoulli  variables,  given  true  score.  An  extension  of  this  example  is  used  below 
to  exemplify  the  technique. 

•  In  an  intelligent  tutoring  system  (ITS),  such  as  the  HYDRIVE  system  for  aircraft 
hydraulics  troubleshooting  (Gitomer,  Steinberg,  &  Mislevy,  in  press),  students 
working  through  a  problem  arrive  at  unique  situations  in  unique  ways,  depending 
on  their  particular  sequence  of  actions.  Their  actions  in  any  situation  are  postulated 
to  depend  on  their  values  of  unobservable  variables  in  a  “student  model,” 
characterizing  key  aspects  of  their  system  and  strategic  knowledge.  It  is  not 
feasible  to  model  every  conceivable  scenario  a  priori  as  a  potential  observable 
variable.  It  may  be  possible,  however,  to  partition  this  large  observation  space  into 
equivalence  classes  defined  in  terms  of  more  abstract  descriptions  of  potential 
outcomes  and  observations,  and  to  specify  corresponding  conditional  probabilities 
given  values  of  student-model  variables.  All  parallel  observations  in  such  an 
equivalence  class  may  then  be  modeled  approximately  as  HOD.  It  will  not  be  known 
a  priori  how  many  observations  in  each  equivalence  class  will  be  realized  for  any 
given  student  in  any  given  problem. 

Our  objective  is  to  carry  out  inference  in  such  situations  without  explicidy  building 
an  inference  network  for  all  the  I ID  variables  in  a  class.  This  can  be  accomplished  if  one 
can  structure  the  join  tree  so  the  nodes  for  variables  in  a  given  HOD  class  appear  only  in 
terminal  nodes,  which  is  straightforward  if  each  such  variable  is  a  terminal  node  in  the 
corresponding  influence  diagram;  i.e.,  it  has  no  children.  This  is  the  case  in  the  settings  in 
which  we  are  interested. 


A  Solution 

The  technique  outlined  below  requires  maintaining  potential  tables  for  only  two 
generic  members  of  the  IID  observation  class  at  any  given  time.  Before  an  observation  is 
made,  both  of  these  tables  embody  the  current  status  of  knowledge  for  die  parent  variables 
and  a  state  of  ignorance  about  the  values  of  the  IID  observations.  When  an  observation  is 
made  on  a  member  of  the  class,  an  inference  engine  (such  as  that  of  ERGO;  see  Mislevy,  in 
press,  for  a  simnple  worked-through  example)  is  used  to  absorb  the  new  evidence  into  the 
rest  of  the  network,  including  the  potential  table  for  second,  not-yet-observed,  generic 
member  of  the  same  IID  class.  The  potential  able  for  the  now-observed  generic  member  is 
replaced  with  a  copy  of  the  updated  able  for  the  not-yet-observed  generic  member.  The 
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resulting  tables  are  (1)  identical  in  structure,  (2)  in  agreement  as  to  die  marginalized 
potential  for  the  parent  variable(s),  and  (3)  in  a  state  of  ignorance  about  the  members  of  the 
IID  variables,  and  thereby  ready  to  repeat  the  process  for  the  next  observation. 

To  specify  this  sequence  more  formally,  let  unknown)  represent  the 

potential  table  at  Time  r  for  the  7th  not-yet-observed,  generic  member  Xj  of  the  HD  class  and 
the  (possibly  multidimensional)  parent  variable(s)  Y.  Also  instantiated  in  the  network  at 
Time  t  is  the  identical  potential  table  for  the  j+  1st  not-yet-observed  member  of  die  IID  class, 
or  f/(y,Xj+i=unknown).  Suppose  the  value  of  Xj ,  say  Xj,  is  now  ascertained.  The  steps 
required  to  update  beliefs  then  effect  the  reduced  computing  representation  of  the  network 
are  as  follows: 

1 .  Update  the  potential  table  containing  Xj,  to  obtain  ft+\(Y J(j=Xj). 

2.  Marginalize  ft+i(YJ(j=Xj)  with  respect  to  Y  to  update  the  clique  intersection  table, 
and  propagate  evidence  throughout  the  rest  of  the  network.  This  includes  obtaining 
fl+\(YJ(j+\=unknown),  which  now  incorporates  a  marginalized  Y  distribution  that 
captures  the  implication  of  Xj=Xj  for  expectations  about  potential  values  of  Xj+\. 

3.  Replace  ft+\(YJCj=Xj)  with  another  copy  of  the  table  ft+j(T,X)+i=unknown).  This 
is  now  thought  of  as  f,+i(K,Xy+2=unknown). 

Thus,  not  all  not-yet-observed  members  of  the  IID  class  need  to  be  represented 
explicitly  because  we  know  from  the  structure  of  the  problem  that  their  potential  tables  are 
identical  to  the  representative  that  is  contained  in  the  network.  That  is,  for  all  n>l, 

ft+l(y,Jfy+n=unknown)  =  f/+i(y,Xy+i=unknown). 

Moreover,  the  potential  tables  for  cliques  corresponding  to  previously-observed  values  of 
X  have  the  same  marginalization  of  Y.  Unless  retraction  of  the  information  about  specific 
values  of  previously-observed  Xs  is  contemplated,  it  is  no  longer  necessary  to  carry  these 
potential  tables  along  in  future  manipulations  of  the  network. 

Example 

This  is  an  example  from  educational  testing,  with  one  variable,  6,  representing 
student  proficiency  and  5  variables,  Itml, ....  Itm5  representing  0/1  (wrong/right) 
responses  to  conditionally  independent  test  items.  Figure  1  is  a  directed  graph 
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representation  of  the  problem.  The  three  possible  values  of  due  (-1, 0,  +1),  with  prior 
probabilities  of  .25,  .50,  and  .25  respectively.  The  conditional  probabilities  of  a  conrect 
response  to  any  item  are  .  10,  .40,  and  .70  respectively.  Figure  2  is  a  join-tree 
representation  of  the  network,  showing  five  cliques  of  the  form  { 6,  Itmj}  connected  to  the 
same  clique  intersection  set  { 6} .  There  are  thus  5  clique  potential  tables  and  one  clique 
intersection  table.  An  ERGO  run-time  file  is  presented  in  the  Appendix. 

[Figures  1  &  2  about  here] 

Table  1  shows  the  updating  of  potential  tables  as  the  response  sequence  (1,1,0,0,1) 
is  absorbed  one  item  at  a  time.  Shaded  areas  represent  potential  tables  for  items  already 
observed.  They  differ  as  to  the  value  of  the  observed  response,  but  agree  as  to  the 
marginal  distribution  of  their  common  parent.  Note  that  at  any  given  point  in  time,  the 
potential  tables  for  all  items  not-yet-observed  are  identical.  Table  2  shows  the  same 
updating,  effected  with  the  “update  &  replace”  strategy  with  only  two  potential  tables  for 
items  at  any  given  point  in  time.  For  example,  after  a  response  of  1  to  Item  1  has  been 
observed,  the  Item  1  potential  table  collapses  to  the  *1=1  column  and  the  potential  table  for 
Item  2  reflects  the  impact  on  belief  about  &,  the  Item  1  table  is  then  discarded,  and  the  Item 
2  table  is  duplicated  to  produce  the  Item  3  table. 

[Tables  1  &  2  about  here] 
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Appendix 

An  ERGO  runtime  file  for  the  sample  problem  is  shown  below,  “q”  stands  far  “ft” 

85227296  6  5  2  6  5  1  3  2  4  3  5  4  6  2  1  2  2  2  2  2  3  0  4  0  0  2  6  2  3  4  5  1  2  2.500000e-02 
2.250000e-01  2.000000e-01  3.000000e-01  1.750000e-01  7.500000e-02  101  3  2  61  13 
1.000000e-01  9.000000e-01  4.000000e-01  6.000000e-01  7.000000e-01  3.000000e-01  1 
0  1  3  2  6  1  14  1.000000e-01  9.000000e-01  4.000000e-01  6.000000e-01  7.000000e-01 
3.000000e-01  1  0  1  3  2  61  1  5  1.000000e-01  9.000000e-01  4.000000e-01  6.000000e-01 
7.000000e-01  3.000000e-01  10  1  3  2  61  1  6  1.000000e-01  9.000000e-01  4.000000e-01 
6.000000e-01  7.000000e-01  3.000000e-01  Itml  ITM2 ITM3 ITM4 ITM5  q  1  0  1  0  1  0  1 
0  10-101 
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Reduced  Representation  Trace  of  Potential  Tables  and  Clique  Intersection  Table 
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